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28 October 1985 


MEMORANDUM FOR: Director of Training and Education 


FROM: 


SUBJECT: Report on the 26 June 1985 Conference of the 

Evaluation Panel on CIA Analysis Training 


STAT 


All members of the Evaluation Panel have approved the 
attached report on the 26 June 1985 Conference, including 
Helene Boatner and Robert Dorn, who did not attend the 
Conference . 



STAT 
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10 September 1985 


MEMORANDUM FOR: 
FROM: 

SUBJECT: 


Director of Training and Education 
Evaluation Panel on CIA Analysis Training 
Report on the 26 June Conference 


SUMMARY OF THE CONFERENCE 

1. The representatives of the Directorate of Intelligence 
and the non— CIA participants in the 26 June conference agreed 
to constitute a continuing Curriculum Evaluation Panel on CIA 
Analysis Training. 

2. The Evaluation Panel strongly endorsed the quality and 
of tne three courses under review: New Analyst Course 

(NAC); Seminar on Intelligence Analysis (SIA) ; and Seminar on 
Intelligence Successes and Failures (ISF). 

3. The principal recommendations by the Panel for 
increasing the impact of the courses on the Directorate were: 

a. That the NAC introduce a unit on Concept Papers, 
include more research-oriented materials and tasks in its 
exercises, and increase the time devoted to computer skills. 

b. That SIA experiment with clusters of students 
from three or four offices or divisions. 


c. That ISF accept less-experienced analysts on the 
recommendation of their division chief. 

4. The next meeting of the panel is scheduled for 
10 January 1986. Because of the concerns expressed by Panel 
members about the training of branch chiefs, the meeting will 
concentrate on training of branch chiefs, especially the 
Supervision of Analysis Seminar. 

THE CURRICULUM EVALUATION PANEL 

The participants at the 26 June conference agreed to 
constitute a continuing Curriculum Evaluation Panel on CIA 
Analysis Training. 
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The Deputy Director for Intell 
members for the Panels 


igence selected the 


following 


Richard Kerr , Associate Deputy Director. 

Helene Boatner . Director, Office of Management, 
Planning and Services.* 


Director, Office 

Chief, National 
vi. rice or boviet Analysis. 


of Global Issues. 
Issues Group, 


John Helge rson , Director, Office of African and 

Latin American Analysis. 

The Director of Training, in conjunction with the Deputy 

Director for Intelligence, selected the non-CIA members of the 
Pane 1 . 


STAT 


t**- • Richard Betts , Brookings Institution. 
Professor Robert Jervis , Columbia University. 
Robert Dorn , Center for Creative Leadership.* 


The following CIA officers have also agreed to serve as 
members of the Panel: 



EFFECTIVE TRAINING 


As indicated in the background paper for the conference (at 
annex to this report), the Director of Training and Education 
has established the Evaluation Panel to strengthen the 
curriculum of courses on analysis training offered on the 
behalf of the Directorate of Intelligence. In opening the 
c ° n ^? rence ' stat ed that his goal was to solicit the advice 
of Directorate managers and outside experts for making an 
already strong program the best of its kind in the country. 


STAT 

STAT 

STAT 


*Did not attend the 26 June confer 
report and agreed to serve on the 


ence, but have approved this 
Panel. 
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The Director of Training and the Evaluation Panel agree that 
achievement of this goal will require regular review of thl 
curriculum to see that m fact the courses individually and 
collectively meet the priority needs of the Directorate in the 
area of analysis training. The effectiveness of the program also 
depends on the availability of high-quality and well-supported 
course directors, who command the respect not only of the 

sponsor^them? Students but also of the Directorate managers who 


EFFECTIVE ANALYSIS AND ANALYSTS 


The first substantive discussion addressed definition of the 
special requirements of and recurring barriers to effective 
analysis that should undergird the curriculum. The Evaluation 
Panel endorsed the list elaborated in the conference paper 

. * _/ Policy Relevance; 2) Ambiguous Information; 

r\ S: .^ X Y e Use of Assum Ptions ; 4) Over- and Under-Confidence- 
->) Clarifying Levels of Confidence; and 6) Alternate Analysis/ 


The DI Panel members, however, produced a list of the 
attributes of an effective analyst , which they believe also 
require priority attention in the curriculum. 


1. Effective command of "tools," 
briefing, and computer skills. 


especially writing. 


2. Readiness to perform the full 
intelligence analyst (e.g., developing 
publishing current intelligence). The 
analysts should be "Perform or Perish," 
Perish . " 


range of roles of an 
data bases as well as 
standard presented to 
not "Publish or 


3. Greater understanding of the substantive review 
process, and command of the skills to facilitate it (e.g 
eliciting and giving feedback). 

4. Ability to use Concept Papers effectively, to 
darify the purposes and audience of assessments, and thus to 
speed the review process. 


-*• Effective relationships with counterparts throughout 
the Agency and Intelligence Community. 

The first two items on the list fall almost exclusively in the 
province of the New Analyst Course. But the list in general will 
serve as another standard for testing the scope and emphasis of 
the syllabi of all analysis training courses. 

For example, the Seminar on Intelligence Analysis can address 
facilitation of the review process in its coverage of utility 
analysis. And the Seminar on Intelligence Successes and Failures 
can emphasize Concept Papers in its coverage of policy relevance. 
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GENERAL EVALUATION OF THE COURSES 

^ The , p f nel “embers present at the conference strongly endorsed 
the quality of the courses under review and their utility for the 
Directorate of Intelligence. The DI managers, while tie? tabled 
most of the specific criticisms, were also thi most out wken in 
expressing appreciation of the program. 

... Dic * Betts and Bob Jervis, the non-CIA panelists who attended 
the conference, were more tentative in both their criticism and 

th^Dn?^ 186 ". refl ® ctin 9 their initial limited exposure to both 
he purposes and performance of the courses. However, they both 
brought to bear their broad experience on the general subject of 
f ^ tlve intell igence analysis, and thereby helped to define some 
^ he . lnh erent tradeoffs confronting the program (e.g., between 
emphasis on basic skills and on creative analysis). 

NEW ANALYST COURSE (NAC) 

The NAC evoked the most attention at the conference — 
testimony to its importance to DI managers during a period of 
unprecedented influx of new analysts. period 

The Panel strongly endorsed the value of the NAC for the 
students and for the Directorate. The Panel agreed that the 
course s length should not be extended beyond its present six 
weeks; and that it should remain a "survival course " 
concentrating on the skills and values new analysts ' need to 
survive their three-year period of probation. 

Bob Jervis noted, however, that the "model" of a single full 
time and relatively short course sets the limits for its^alues 
within which only marginal improvements can be sought. The DI has 
rejected two alternate models which would provide different 
values: a series of part-time courses tailored to the needs of 

Trai nlna^ 1 £ ec £ ults; and a prolonged program (such as Career 

wide-ranging ‘needs^^ Pr ° Vlde tlme £ ° r the COVerin 9 “ese 

The following specific recommendations for the NAC were 
raised, mostly by the DI Panelists. 
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1. Concept Papers. Too many draft assessments still show 
a diffuseness of purpose that encumbers the review process. 

New analysts must tailor their assessments to a specific 
audience and purpose. Concept Papers constitute essentially a 
contract between analysts and managers, and the NAC should 
introduc e a unit on th e value of and formats for Concept 
Papers. [ ~ ~ | who spoke most forcefully to the need for 

such a unit kindly volunteered to present it at the next 
running of the course. 


2. Computer Skills. ADP competency will be essential to 
the future functioning of the Directorate, and an expansion of 
the present two and one— half days of instruction will probably 
be needed as this increased dependency develops. In fact, the 
NAC now includes five days on ADP. 

3. The NAC should increase its attention both to research 
non— pol i t ical analysis. The unit on source 

familiarization, for example, can address data bases as well as 
time-sensitive traffic. And the writing drills can include an 
article for the weekly economic serial. An exercise on 
conventional military analysis has already been added. 

4. In this context, of broadening the definition of what 
is important to the Directorate and therefore career enhancing 
for new analysts, the Panel recommended that the standard of 
"Publish or Perish" be replaced with "Perform or Perish." 

Toward this end, new analysts should be instructed that they 
are expected to develop competency in all aspects of 
intelligence analysis: e.g., the development of data bases as 
well as policy-relevant current analysis. 

SEMINAR ON INTELLIGENCE ANALYSIS (SIA) 

The Panel also strongly endorsed SIA. Both the student 
participants and the sponsoring office managers see the course 
as providing substantial and distinctive values in the form of 
greater capability to apply discipline (structure) and 
creativity to analytical assignments. 

The main concern was how to cope with the analysts' 
perception of the lack of positive organizational incentives 
for applying the techniques learned in the course. No one 
could come up with a credible definition of the source of the 
resistance — although it was suggested it might be the residual 
influence of long-retired "city-room" supervisors. 

In a post conference discussion of the problem of 
resistance to change. Panel member Bob Dorn recommended that 
division chiefs be tasked to provide the needed incentives, 
since they can afford to take more risks than branch chiefs. 


STAT 
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The Panel also recommended increasing the post-course 
impact by structuring each class with clusters of four or five 
students from three or four divisions or offices. This 
arrangement will be attempted in the near future — in order to 
provide some commonality in substantive specialities (not 
available when the students represent all the DI offices) and 
also some diversity in organizational dynamics (not available 
when the class represents a single office). 

The Panel agreed that SIA should continue the recent trend 
of accepting analysts with three to five years of experience 
(rather than the old norm of seven or more years). 

One of the DI Panel members indicated that SIA should 
nonetheless still be seen as a course for a limited number of 
analysts. The non-CIA members, in contrast, thought that all 
or most analysts should be exposed to the values of the course. 

SEMINAR ON INTELLIGENCE SUCCESSES AND FAILURES (ISF) 

The Evaluation Panel also strongly endorsed ISF as 
presently constituted, in terms of the beneficial impact on the 
students and on the Directorate. 

One of the DI members suggested that junior analysts, on 
the recommendation of their division chiefs, be accepted for 
the Seminar. 

THE NEXT MEETING OF THE EVALUATION PANEL 

The next meeting of the Evaluation Panel is tentatively 
scheduled for the 10 January 1986. 

During the 26 June conference) concern about the 
development and training of branch chiefs was repeatedly raised 
as a priority concern in the Directorate. Because of a 
relative dearth of senior analysts, new branch chiefs are being 
selected with less experience than in the past. Moreover, 
there are fewer experienced analysts in their units to share 
the responsibility for supervision and on-the-job training. 

Consequently, the January meeting of the Evaluation Panel 
will address the Supervision of Analysis Seminar — a two week 
course for new branch chiefs, as well as other OTE supervisory 
courses . 

At the same time, the Panel could discuss what other 
services the Office of Training can provide to address the 
general problem of relatively inexperienced and overtaxed 
supervisors. Perhaps arrangements can be made for course 
directors and DI annuitants on contract to OTE to visit the 
branches periodically to assist with on-the-job training 
(circuit-riding instructors). 
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